Conversion of interleaved data sets, including chroma correction and/or correction of checkerboard interleaved formatted 3D images

ABSTRACT

Conversion of interleaved data and/or correction of color samples or other data are corrected by recognizing an underlying arrangement or format of different data sets within a data stream and a conversion process applied that causes bleeding between the sets. The data sets are, for example, separate channel views of a 3D display, and the corruption occurs, for example, upon up-conversion of color samples that take into account both views together rather than individually. The invention is embodied, for example, as part of a playback device, display, or as a stand alone converter box that corrects the corrupted samples by at least one of substitution, filtering, or interpolation with appropriately selected samples (e.g., neighboring samples of a same view).

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Patent Provisional ApplicationNos. 61/191,416, filed 7 Sep. 2008, and 61/143,087, filed 7 Jan. 2009,hereby incorporated by reference in their entireties.

COPYRIGHT NOTICE

A portion of the disclosure of this patent document contains materialwhich is subject to copyright protection. The copyright owner has noobjection to the facsimile reproduction by anyone of the patent documentor the patent disclosure, as it appears in the Patent and TrademarkOffice patent file or records, but otherwise reserves all copyrightrights whatsoever.

BACKGROUND OF THE INVENTION

1. Field of Invention

The present invention relates to data encoding and decoding, conversionof data with respect to a format or arrangements of the data, and thecorrection of errors from conversions without appropriate considerationof a format or arrangement of the converted data.

2. Description of Related Art

In recent years, content providers have become interested in thedelivery of stereoscopic (3D) content into the home. This interest isdriven by the increased popularity and production of 3D material, butalso the emergence of several stereoscopic devices that are alreadyavailable to the consumer. Although several systems have been proposedon the delivery of stereoscopic material to the home that combinespecific video view “arrangement” formats with, primarily, existingvideo compression technologies such as ISO MPEG-2, MPEG-4 AVC/ITU-TH.264, and VC-1, these systems do not provide any information on how thevideo encoding process should be performed. This has consequentlyresulted in poorly designed stereo video encoding solutions with subparperformance, which has been detrimental in the adoption of such systems.

SUMMARY OF THE INVENTION

The present inventors have realized the need to provide a system for theprovision of a stereoscopic (3D) user experience implemented withminimal or no alteration to existing playback devices such as set-topboxes, DVD, and Blu-ray disk players, as well as existing 3D capabledisplays. This comprises the provision of either an appropriate colorconversion considering a format in which the 3D images are provided in avideo stream or a device capable of correcting errors if the conversionis done without appropriate consideration of the format. The inventionmay be embodied in multiple forms as further described herein and may bepart of a playback device, a display, set-top type converter box, and/orother embodiments as described herein.

Portions of both the device and method may be conveniently implementedin programming on a general purpose computer, or networked computers,and the results may be displayed on an output device connected to any ofthe general purpose, networked computers, or transmitted to a remotedevice for output or display. In addition, any components of the presentinvention represented in a computer program, data sequences, and/orcontrol signals may be embodied as an electronic signal broadcast (ortransmitted) at any frequency in any medium including, but not limitedto, wireless broadcasts, and transmissions over copper wire(s), fiberoptic cable(s), and co-ax cable(s), etc.

BRIEF DESCRIPTION OF THE DRAWINGS

A more complete appreciation of the invention and many of the attendantadvantages thereof will be readily obtained as the same becomes betterunderstood by reference to the following detailed description whenconsidered in connection with the accompanying drawings, wherein:

FIG. 1 is an illustration depicting a pixel arrangement for acheckerboard interleaving scheme, and the subsequent deinterleaving;

FIG. 2 is an illustration of pixel replication performed via a genericupsampling;

FIG. 3A is a block diagram of a system 300 incorporating a converter box(3D Color Correction) 330 according to an embodiment of the presentinvention;

FIG. 3B is a block diagram of a system incorporating a playback device380 according to an embodiment of the present invention;

FIG. 3C is a block diagram of a system incorporating a display deviceaccording to an embodiment of the present invention;

FIG. 4 is an illustration of a basic process that performs colorcorrection according to an embodiment of the present invention;

FIG. 5 is an illustration of a serial type sample replacement accordingto an embodiment of the present invention; and

FIG. 6 is a block diagram illustrating an embodiment where two views aredeinterleaved, upscaled from CB to full resolution and then displayed ontwo display devices using parallel HDMI interfaces.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Exemplary 3D displays are, for example, recently developed DLP andPlasma displays by Samsung and Mitsubishi, which will benefit from asystem according to the present invention for the provision of astereoscopic (3D) user experience. Other displays will eventually beoffered in need of the same or similar capabilities.

One format for the delivery of 3D content is to multiplex the two viewsusing a checkerboard (CB) arrangement. Although this arrangement can bedone using different color spaces or color configurations, because oflimitations in existing devices, creation and encoding is typicallyperformed in the YCbCr colorspace using, for example, a 4:2:0 colorsampling process.

Delivery of 3D content using the checkerboard method impliesinterleaving of the left and right view images using a checkerboard (CB)arrangement (or format). Although this arrangement may complicate andimpact the encoding process, it helps to better preserve the frequencycharacteristics of the 3D signals compared to other comparabletechniques such as interleaving the 3D views in a row or columninterleaved arrangement or placing the views side by side. The pixelarrangement for the checkerboard interleaving scheme, and the subsequentdeinterleaving that will be performed at the display device, aredepicted in FIG. 1A.

For some display devices, such as existing DLP TVs, the deinterleavedimages will be displayed as is exploiting the characteristics of thedisplay mechanisms, while in others, such as Plasma displays, additionalinterpolation/upsampling may be performed in order to bring the image toits full, original resolution. Ideally, if no compression or otherconversions are performed onto the original, CB source, no blending ofthe information from the original left and right views in the final,deinterleaved, left and right views at the display is expected. Inreality, however, because of lossy compression information leakage fromone view to another may occur. This could result in undesirableartifacts during 3D display. Nevertheless, this problem can beconsiderably constrained and controlled by introducing a variety ofconsiderations into the encoder such as avoiding or negatively biasingcertain coding tools, e.g. deblocking filter, subpixel motion positionsetc. (see, for example, Tourapis et al, U.S. patent application Ser. No.13/054,912, the contents of which are incorporated herein by referencein their entirety).

Unfortunately, problems may happen in the case that either the videosource is upconverted in the decoding device or display, or if color andformat conversion is requested without any considerations of the formatarrangement. The first may result in completely damaging thecharacteristics of the signal, while the later may result in colorbleeding problems from one view to the other especially around regionswith high view disparity characteristics. Color bleeding typicallyoccurs because chroma information is usually subsampled and signaled ata lower resolution than luma information for video compression and thedelivery or display mechanism requires a completely different chromasubsampling or even color format to be used.

More specifically, YUV 4:2:0 subsampling is most commonly used in whichthe chroma components are subsampled to half the horizontal and verticalresolution. The location of the chroma samples relative to the lumasamples of the video frame may vary. This format is mandated by manyvideo coding systems, including applications for video content deliveryto the home using broadcasting and digital video recording (DVD,Blu-ray, HD-DVD etc) media. In the case of YUV 4:2:2 subsampling, onlythe horizontal resolution of the chroma is halved and each chroma samplecorresponds to two horizontally adjacent luma samples. Other less commonarrangements such as YUV 4:1:1, 4:2:1, 4:1:0, 8:4:4, 3:1:1 and possiblyothers could also be encountered. It is also possible to encounter caseswith no subsampling, i.e. 4:4:4 Y′CbCr and R′G′B′ sampling.

Delivery, however, can be constrained by the delivery interface, e.g.High-Definition Multimedia Interface (HDMI) or Digital Visual interface(DVI), which may impose certain characteristics on the color samplingand color format. In particular, the HDMI interface only supportsoutputs in xvYCC 4:4:4, sRGB 4:4:4, YCbCr 4:4:4, and YCbCr 4:2:2, whileexisting 3D enabled displays, such as several Samsung DLPs, require that3D inputs are formatted as sRGB 4:4:4. However, since the originalcheckerboard source material is formatted as YCbCr 4:2:0, a conversionprocess, that includes upscaling the chroma components (e.g., usingsimple pixel replication) and color conversion is utilized.Unfortunately, if no consideration is made for the nature of thecontent, i.e. checkerboard format or interleaving of the two views,during upconversion, this may result in severe color artifactsespecially around objects with high disparity.

An example of such a scenario where color information from the rightview has “leaked” into the left view would be where an object exhibits acolor bleeding artifact. Such artifacts will become apparent in 3Dimagery by a shadow at a border of the objects. For example, a 3-D imageof a glove may include, for example, an undesirable color bleedingapparent with a reddish shadow just past the border of the glove. Theseartifacts are most apparent at boundaries but are also included the fullview and are better explained by showing how the bleeding problemoccurs.

This bleeding problem is primarily inherent of the upscaling processperformed in almost all of the existing systems. In particular, if theupconversion of the chroma components does not account for content thatis checkerboard formatted, it would try to upscale the chroma samplesusing simple filtering mechanisms that consider all samples from bothviews. In the simplest and most commonly used method, pixel replicationis performed as shown in FIG. 2 (generic upsampling). In this scenario,it can be seen from FIG. 2 that when deinterleaving the new sequence togenerate the two separate views for display (left view samples and rightview samples), half of the pixels in each view would be incorrectlyassigned with color samples corresponding to the opposite view. Theproblem may be somewhat aggravated if color conversion is also performedbecause of clipping and rounding operations. It is possible that use ofan advanced interpolator, all of the samples could be incorrect due tothe filtering.

The problem occurs to a greater or lesser extent regardless of theinterleaving pattern or the upconversion process utilized, unless, bydesign or coincidence, the upconversion process accounts for theinterleaving of the views. The problem also occurs regardless of whetherthe views are 3D views or entirely different images, video, or otherdata. The problem is likely exasperated if the images are entirelydifferent (e.g., separate programs) because the images have nosimilarity (e.g., instead of an artifact that, for example, shadows theintended image, the artifact is likely of an entirely different shape).

To resolve this problem, the present invention provides severalembodiments for solutions. For example, given that already availabledevices have this problem, a separate converter box may be used thattries to correct this issue. This box can be placed at the output of theplayback/decoder device and after any necessary decryption or otherprocessing of the signal (e.g. HDMI), can convert the input signal(e.g., a 3D checkerboard multiplexed format with improper upconversionand therefore color or other artifacts) to a new signal appropriatelycorrected and repackaged (e.g. HDMI) and re-encrypted. This new signalcould contain additional metadata information that could help drive orseamlessly control next generation 3D capable displays that canrecognize these signals. In general, the output of the converter willthen be connected to a 3D capable display, but could also involve otherdevices, such as a multimedia receiver, an HDMI bridge, HDMI to DVI orvice versa converters etc, prior to reaching a display, or some othermedium. The converter box may include, for example, a pattern recognizerthat searches for known patterns of color errors and then applies a mostappropriate schema for correction. Such recognition may comprise, forexample, comparing diagonal patterns of chroma samples in one viewagainst diagonal patterns of chroma samples in an opposite view. A closematch on the opposing diagonals may trigger the color correctionprocesses. The device may include a database of patterns from which torecognize and then apply an appropriate correction schema. Each patterncomprises, for example, a pattern of errors associated with acombination of an interleaving arrangement (or format) and a conversion(e.g., up-conversion) and is associated with a schema specificallyderived to correct or otherwise minimize errors occurring due to theinterleaved/conversion combination.

FIG. 3A is a block diagram of a system 300 incorporating a converter box(3D Color Correction) 330 according to an embodiment of the presentinvention. The system starts by encoding left and right images whichincludes, for example interleaving the left and right images. A playbackdevice 320 decodes the interleaved images and, for example, performs anupconversion/conversion of the chroma color space (e.g., an upconversionas described above or a variant thereof).

The converter box (3D color correction) 330 performs a process thatinvolves performing colorspace conversion, if necessary, back to itsoriginal (e.g., YCrCb), replacement of incorrect color samples withsamples derived from the same view (e.g., using pixel replication ormore advanced filtering methods, including temporal interpolation suchas a motion compensated or motion adaptive temporal filter, whileconsidering samples only from the same view), followed by colorspaceconversion to the desired colorspace and/or format (i.e. sRGB 4:4:4).

It should be noted that in various likely implementations, the converterbox 330 would require, among other things an HDMI receiver and encoder,which also involve encryption/decryption licensing and would furtherincrease overall cost. Note that this converter box may also optionallyadd additional metadata information that could help with the playbackand conversion of the 3D data in future displays.

A deinterleave device then extracts or otherwise builds the left andright images to be displayed by selecting appropriate portions of thecolor corrected and decoded, but interleaved images. The deinterleavedevice is, for example, a checkerboard deinterleaver, but may also beconfigured to operate on images interleaved in any of line-by-line,side-by-side, or other interleaving formats.

The above process (or others as described herein) could be addeddirectly on the display (e.g., display 390 in FIG. 3C). This avoidshaving to add additional HDMI related hardware. If a display is designedwith programmable chips, the provision of further color conversionmechanisms would be facilitated (alternatively as a retrofit).Alternatively, the sets are redesigned with new hardware specificallyfor these tasks.

In one embodiment, a “proper” chroma upconversion method is included inthe decoder system. This is apparently the “best” solution especiallysince it avoids difficult to reverse problems due to clipping orrounding, reduces processing (essentially replaces the originalupsampling process which needs to be performed regardless of theconversion), and, again, does not add any additional cost due to HDMIlicensing.

The decoder system may be, for example, the display itself (built intothe display), a set-top box (cable box, satellite box, converter box,etc), DVD player, Blu-ray players, or any playback device (e.g., seeplayback device 380 in FIG. 3B), as well as existing 3D capabledisplays. While existing 3D display will generally reap the greatestimmediate benefit from such devices/improvements, the invention is notlimited to supplying such devices, as the interleaved formats may beutilized for the provision of content to standard displays (see below),and newly developed technologies will clearly benefit from theimprovements in content delivery as elaborated on herein.

FIG. 4 illustrates a basic process that performs color correction, aswould be performed, for example, in the converter box 330. This processperforms a horizontal replacement of incorrect view samples usingcorrect horizontal only samples. For example, a left color sample value410 in the Cr space that has corrupted the right color samples isreplaced by a neighboring right sample Cr (e.g., replace with eithersample 405 to its left or sample 415 to its right).

Diagonal or/and vertical replacement filtering of appropriate same viewsamples could also be considered. In alternative embodiments, averaging,filtering, extrapolation, or interpolation are utilized to determine thereplacement chroma value.

The same process, using only sample replacement, is illustrated in FIG.5 assuming serial processing as is likely to be performed on the HDMIbased converter box. Such an embodiment uses, for example, a buffer thatstores N samples. The number of samples stored in the buffer woulddepend on the number of samples used in a filtering scheme and on howfar apart the samples may be (e.g., diagonal replacement may requiremore stored samples than a neighboring sample replacement).

As shown in FIG. 5, left (X) and right (Y) samples of an interleaved 3Dimage are serially output from a playback device. The samples are storedin a buffer and then replaced. Sample 510 is an uncorrupted left sample,sample 520 is a right sample having a corrupted chroma portion, sample530 is a left sample having a corrupted chroma portion, and sample 540is an uncorrupted right sample. As shown, the Cb and Cr portions of thecorrupted samples are replaced with the Cb and Cr from their neighboringuncorrupted samples. This assumes that signal is YCbCr. If signal isRGB, then all samples should be reverted back to YCbCr color space,perform correction, and then perform color conversion again. This isstill a replacement process but it is a replacement process done in thecolor transform domain.

The replacement samples comprise, for example, the nearest neighboringuncorrupted sample of the same view (left corrupted samples are replacedwith neighboring, if available, left uncorrupted samples). As noted,alternative schemes including averaging, filtering, interpolating, orextrapolating from one or more (perhaps several) neighboring samplescould be performed. And, the samples chosen can be neighboringhorizontal, vertical, or diagonally related to the replaced sample. Someembodiments might even consider the effect of samples further than justthe neighboring samples in a multi-sample filter or weighted averagingscheme.

Again, assuming that the colorspace is the same as that of the originalsource, correction can involve a simple replacement, or some otherfiltering process of the incorrect view samples in the chroma componentswith those of neighboring samples from the same view. In a certainembodiment only horizontal samples are considered and the process isperformed using bilinear filtering. In a different embodiment, simplereplacement is performed with either the nearest neighboring sample fromthe same view, or from a sample always in the same direction. In anfurther embodiment different or all directions could be considered, anda variety of filters could be used to generate the missing samples. Thiscould provide a variety of complexity (memory, speed, delay) andperformance (interpolation and image quality) tradeoffs.

If, however, a different colorspace than that of the original contentwas used, e.g. RGB or xvYCC, XYZ among others, the process may alsoinvolve apart from sample correction a color correction process. Thiscould be done using either a multistage approach where one firstreconverts the samples in error to the original color space, determinesthe correct color values in the original space given the above method,and then performs color conversion to the required space. In a differentembodiment, color conversion could be performed in a single step bycombining neighboring samples. For example, assuming that colorconversion from one color space to the other and vice versa is performedas:

$\begin{bmatrix}C_{0} \\C_{1} \\C_{2}\end{bmatrix} = {{{\begin{bmatrix}a_{00} & a_{01} & a_{02} \\a_{10} & a_{11} & a_{12} \\a_{20} & a_{21} & a_{22}\end{bmatrix}\begin{bmatrix}A_{0} \\A_{1} \\A_{2}\end{bmatrix}} + {\begin{bmatrix}b_{0} \\b_{1} \\b_{2\;}\end{bmatrix}\mspace{14mu}{{and}\begin{bmatrix}A_{0} \\A_{1} \\A_{2}\end{bmatrix}}}} = {{\begin{bmatrix}c_{00} & c_{01} & c_{02} \\c_{10} & c_{11} & c_{12} \\c_{20} & c_{21} & c_{22}\end{bmatrix}\begin{bmatrix}C_{0} \\C_{1} \\C_{2}\end{bmatrix}} + \begin{bmatrix}d_{0} \\d_{1} \\d_{2\;}\end{bmatrix}}}$

and C_(x) ^(T) are neighboring samples without color problems, C_(x)^(B) samples for the current pixel with problems, then one could computethe corrected samples C_(x) ^(C) as:

$\begin{bmatrix}C_{0}^{C} \\C_{1}^{C} \\C_{2}^{C}\end{bmatrix} = {{\begin{bmatrix}a_{00} & a_{01} & a_{02} \\a_{10} & a_{11} & a_{12} \\a_{20} & a_{21} & a_{22}\end{bmatrix}\begin{bmatrix}{{c_{00} \times C_{0}^{B}} + {c_{01} \times C_{1}^{B}} + {c_{02} \times C_{2}^{B}} + d_{0}} \\{{c_{10} \times C_{0}^{T}} + {c_{11} \times C_{1}^{T}} + {c_{12} \times C_{2}^{T}} + d_{1}} \\{{c_{20} \times C_{0}^{T}} + {c_{21} \times C_{1}^{T}} + {c_{22} \times C_{2}^{T}} + d_{2}}\end{bmatrix}} + \begin{bmatrix}b_{0} \\b_{1} \\b_{2}\end{bmatrix}}$

Different color transforms could be considered accordingly, while it ispossible that a completely different color transform and color samplingmay be desired altogether. Appropriate color format selection andnecessary color correction can be performed given knowledge of theoriginal input signal, the current formatting of the content, and thefinal output device, e.g. display capabilities.

Since in certain interfaces, such as HDMI, the signal will be receivedand decoded serially by the color correction module, in one embodiment,the memory requirements as well as the latency introduced by the colorcorrection module can be minimized by performing the color correctionalso in a serial fashion. As illustrated in FIG. 5, the color correctionmodule only requires one neighboring pixel of the same parity view toperform a color correction on a given pixel (i.e., only 3 pixels need tobe buffered). In this case, the chroma components, which are known to becorrect, of the nearest neighboring pixel of the same parity, are copiedto replace the incorrect chroma components of the current pixel. Colorcorrection can be performed, if necessary, as described previously.

Further refinements that improve the color reconstruction but requirethe buffering of a larger number of pixels are also possible at the costof increased latency and memory requirements. In a certain embodimentcolor correction is performed given previously received samples (i.e.causal color correction), or also consider N additional future samples.Future samples could be also constrained to be only on the same row oralso allow M additional rows to be considered. In a further embodiment,color correction and interpolation of the samples could consider theentire picture, or even temporal samples from previous or past pictures.For this purpose one may consider, for example, only co-located,temporally, samples, assuming no view parity change, or also considermotion adaptive or motion compensated methods. View parity changes canalso be considered accordingly.

Assuming that the interface of interest is HDMI, or somethingequivalent, and that a correction device is to be implemented, then thisdevice may also have to perform decryption and encryption operationssuch as the High-bandwidth Digital Content protection HDCP. In a certainembodiment, decryption and optionally encryption are performed beforeand after color format conversion and color correction. It is alsopossible to signal the existence of 3D content, the color formatsampling that the content is currently in, and if the current formatrequires any additional color correction prior to display.

Furthermore, it is possible to also signal optional postprocessing,ideal interpolation filters for chroma but also for 3D view upsampling,e.g. such as the one required by plasma displays to currently properlydisplay checkerboard interleaved video signals, by using additionalcontrol information available in such interfaces. Information could besignaled separately for each view since their characteristics may bequite different. In particular, the HDMI specification allows suchinformation to be provided within the vendor specific data block, whileit may be possible to also provide similar information at a higherlevel, i.e. within a bitstream using other metadata transmissionmechanisms, e.g. MPEG-4 or MPEG-2 systems metadata, SEI messages withinan H.264/MPEG-4 AVC bitstream etc. In a certain embodiment, an inputdevice is appropriately initialized given this metadata information, andcan automatically, or selectively given user interaction, process thecontent for optimal delivery to the user.

In various embodiments, the color formatting, upscaling, and correctionmodule can be implemented within the decoder prior to transmission ofthe signal to the display device, or alternatively within the displaydevice. Optionally, in the display device, the input signal can berestricted to a single format, i.e. RGB 4:4:4 which would only requirecolor correction to be performed reducing the cost and foot print of thedevice. In both scenarios, additional HDMI handling may be unnecessarysince such is already handled by the device itself.

Using existing mechanisms for rescaling (downscaling and upscaling) CBinterleaved video content would result in damaging the 3D nature of thecontent. To alleviate this problem, apart from the above mechanismsintroduced for chroma correction, the present invention also introducesrescaling for CB content by considering its characteristics. Morespecifically, instead of performing rescaling using all samples in aneighborhood of a pixel, only samples belonging to the same view areconsidered for rescaling. Furthermore, view samples are generated againby considering their sample positions in the CB placement as to ensureproper 3D content reconstruction. In a certain implementation this canbe done by first deinterleaving the content to generate the separateviews, optionally replacing the missing samples, and then resampling thetwo views separately. Finally the images are interleaved using the CBpattern once more. In an alternative embodiment, this process could bedone without performing deinterleaving but appropriately designingfilters for each pixel position given its view correspondence (i.e.assuming separable filters these could be of the form [a2 0 a1 0 a0 0 a10 a2] etc).

The above description could be generalized for signals comprising ofmultiple views more than 2, or signal comprising of two or morecompletely separate/independent image views (non 3D) that may have beeninterleaved together for purposes such as exploiting the characteristicsof a display or other applications (e.g., multi-player video games,driver/passenger display with different content option etc).

One embodiment involves allowing the system to select one of the viewsthis could be done by the user or maybe by other metadata includedsomewhere else in the chain, deinterleaving this particular view,interpolating this view and only this view and then transmitting thatview to a display device. The selection of the view could be done at thestart of the sequence, every GOP, or adaptively given the signaling ofthe metadata information. This process would enable non 3-D displays tostill display the content in a 2-D mode enabling backwards displaycompatibility. In other embodiments, the views interleaved are not 3-Dviews, but instead entirely different images or video sequences.

In one embodiment, as shown in FIG. 6, a device according to the presentinvention is configured such that both views are deinterleaved, upscaledfrom CB to full resolution and then, using two “parallel” HDMI channelssend to different displays or send to a display that uses a different 3Ddisplaying method that assumes the use of full resolution images. In analternative, these two upconverted views are again re-multiplexed.Multiplexing can still occur using the checkerboard method. Theprovision of two HDMI (or other) outputs and the correspondingfunctionality may also be provided in a different arrangement of formatsfor the two images (i.e. full left view first, right view second, orleft view row first, right view row second etc).

Although the present invention has been described herein with referenceto 3D views interleaved, encoded, decoded, and corrected, it shouldagain be clarified that the present invention does not require, norshould it be limited to 3D views, or views at all, and is more generallydescribed as the color format/sampling conversion and/or, if needed,correction of data errors due to some type of conversion that fails totake into account the interleaved format of the data sets. The presentinvention may be applied to any data stream using any type ofinterleaving method (checkerboard, line-by-line, etc.). The devices andprocesses of the present invention may be applied to data sets of anytype or mixed types, including audio or general data information.

In describing preferred embodiments of the present invention illustratedin the drawings, specific terminology is employed for the sake ofclarity. However, the present invention is not intended to be limited tothe specific terminology so selected, and it is to be understood thateach specific element includes all technical equivalents which operatein a similar manner. For example, when describing an HDMI interface, anyother equivalent device, or any device having at least part of thefunctionality of an HDMI device should be considered, such as, forexample, DVI, RGB, or other data/information interfaces or any otherdevice having an equivalent function or capability, whether or notlisted herein. Furthermore, the inventors recognize and havespecifically stated that newly developed technologies not now known mayalso be substituted for the described parts and still not depart fromthe scope of the present invention. All other described items,including, but not limited to interfaces, buffers, conversion processes,interleaving arrangements (or formats), coding, decoding, and displaytechniques, etc should also be considered in light of any and allavailable equivalents.

Portions of the present invention may be conveniently implemented usinga conventional general purpose or a specialized digital computer ormicroprocessor programmed according to the teachings of the presentdisclosure, as will be apparent to those skilled in the computer art.

Appropriate software coding can readily be prepared by skilledprogrammers based on the teachings of the present disclosure, as will beapparent to those skilled in the software art. The invention may also beimplemented by the preparation of application specific integratedcircuits or by interconnecting an appropriate network of conventionalcomponent circuits, as will be readily apparent to those skilled in theart based on the present disclosure.

The present invention includes a computer program product which is astorage medium (media) having instructions stored thereon/in which canbe used to control, or cause, a computer to perform any of the processesof the present invention. The storage medium can include, but is notlimited to, any type of disk including floppy disks, mini disks (MD's),optical discs, DVD, HD-DVD, Blue-ray, CD-ROMS, CD or DVD RW+/−,micro-drive, and magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs,DRAMs, VRAMs, flash memory devices (including flash cards, memorysticks), magnetic or optical cards, SIM cards, MEMS, nanosystems(including molecular memory ICs), RAID devices, remote datastorage/archive/warehousing, or any type of media or device suitable forstoring instructions and/or data.

Stored on any one of the computer readable medium (media), the presentinvention includes software for controlling both the hardware of thegeneral purpose/specialized computer or microprocessor, and for enablingthe computer or microprocessor to interact with a human user or othermechanism utilizing the results of the present invention. Such softwaremay include, but is not limited to, device drivers, operating systems,and user applications. Ultimately, such computer readable media furtherincludes software for performing the present invention, as describedabove.

Included in the programming (software) of the general/specializedcomputer or microprocessor are software modules for implementing theteachings of the present invention, including, but not limited to,substituting chroma samples, averaging samples, interpolating samples,extrapolating samples, filtering samples including averaging andweighted averaging multiple samples, recognizing patterns of corruptedsamples, applying correction schemas, recognizing user or other inputsto direct 2-D or 3-D displays, and the display, storage, orcommunication of results according to the processes of the presentinvention.

The present invention may suitably comprise, consist of, or consistessentially of, any of element (the various parts or features of theinvention) and their equivalents as described herein. Further, thepresent invention illustratively disclosed herein may be practiced inthe absence of any element, whether or not specifically disclosedherein. Obviously, numerous modifications and variations of the presentinvention are possible in light of the above teachings. It is thereforeto be understood that within the scope of claims to be included in asubsequently filed utility patent application, the invention may bepracticed otherwise than as specifically described herein.

By way of further examples, in various embodiments, the inventioncomprises, and may be embodied, as, for example:

The invention claimed is:
 1. A method, comprising the step of correctinga sample error in at least one pixel of a first image in a video stream,wherein the video stream comprises a checkerboard arrangement of twoviews of 3D video content, said video stream comprising a firstcolorspace and the sample error comprises color bleeding from a pixel ofa second image in the video stream, wherein the color bleeding comprisesbleeding induced by an up-conversion of a frame containing the firstimage and the second image, the method comprising: performing a colorconversion to a second colorspace; applying a correction schema by whichat least one incorrect color sample in the first image is replaced by areplacement color sample derived from at least one correct color sampleof the same image; wherein the correction schema comprises a schemaretrieved from a database of schemas; and wherein the step of correctingcomprises substituting an incorrect color sample in the first image by acorrect neighboring color sample of the same image.
 2. The methodaccording to claim 1, wherein the up-conversion comprises a color spaceconversion.
 3. The method according to claim 1, wherein applying thecorrection schema comprises recognizing a known pattern of color errorsin the frame and applying a specific correction schema associated withthe recognized pattern of errors to correct or minimize the errors. 4.The method according to claim 1, wherein the images are left and rightviews of a stereo pair.
 5. The method according to claim 1, furthercomprising the step of recognizing an up-converted interleaved videostream and enabling the step of correcting.
 6. A correction device,comprising: a correction mechanism configured to correct color errors ina video data stream, the video data stream comprises a first colorspaceand a checkerboard arrangement of two views of 3D video content; whereinthe errors comprise data bleeding between different portions of the datastream due to an operation performed on the data stream not intended fora format of the different portions; wherein the different portionsrepresent separate images each comprising left and right views of a 3Dimage; wherein the separate images are images of a same frame in thevideo stream, and the operation comprises a video image formatconversion; wherein the correction device is configured to perform acolor conversion to a second colorspace and apply a correction schemaafter identifying a need for correction via comparison of differentviews of the same image, said correction scheme in which at least oneincorrect color sample in one of the separate images is replaced by areplacement color sample derived from at least one correct color sampleof the same image; a recognition device configured to recognize an errorpattern in the data stream; and a set-up device configured to set up thecorrection mechanism to correct the recognized error pattern, saidcorrection mechanism configured to interrogate a database of correctioninformation and determine a best correction schema to be applied by thecorrection mechanism; the correction device comprising a converter boxconnected to a display configured to display the images.
 7. Thecorrection device according to claim 6, further comprising a bufferconfigured to temporarily store N data words from at least one portionof the data stream near an error; wherein the correction mechanism isfurther configured to correct the error via one of substitution,filtering, interpolation, and extrapolation of at least one of the Ndata words.
 8. The correction device according to claim 6, furthercomprising a selection device configured to select a correction schemafor the error correction.
 9. A system, comprising: a media sourcecomprising a decoder configured to decode a video stream, the videostream comprising a first colorspace and a checkerboard arrangement oftwo views of 3D video content and an up-converter configured toup-convert the decoded video stream; a correction device configured tocorrect conversion errors in the up-converted decoded video stream;wherein the conversion errors comprise bleeding between two separatedata portions within a frame of the up-converted decoded video stream;wherein the separate data portions represent separate images; whereinthe correction device is configured to perform a color conversion to asecond colorspace and apply a correction schema by which at least oneincorrect color sample in one of the separate images is replaced by areplacement color sample derived from at least one correct color sampleof the same image; wherein the correction schema comprises a schemaretrieved from a database of schemas maintained in a playback devicehaving at least one output to a display; a selection mechanismconfigured to accept a selection comprising one of the separate imagesand cause the selected image and any corresponding images within thestream to be output after correction, wherein the selected image andcorresponding images comprises one channel of a pair of 3D images, andwherein correction is applied after determining identifying a need forcorrection via a close match on opposing diagonals of image data of thecorresponding images.
 10. The system according to claim 9, whereinapplying the correction schema comprises recognizing a known pattern ofcolor errors in the frame and applying a specific correction schemaassociated with the recognized pattern of errors to correct or minimizethe errors.
 11. The system according to claim 9, wherein the decoder issignaled using metadata in the video stream.
 12. The system according toclaim 9, wherein the two separate portions comprise left and rightchannels of a pair of 3D images.
 13. The system according to claim 9,wherein data values of the separate images are interleaved within theframe.
 14. The system according to claim 13, the display configured toreceive the frame containing the separate images, and further configuredto receive the separate images via separate input channels.